Materiel  Test  Procedure  6-1-001 
Electronic  Proving  Ground 

U.  S.  ARMY  TEST  AND  EVALUATION  COMMAND 
BACKGROUND  DOCUMENT 


TESTING  COMMUNICATION,  SURVEILLANCE 
AND  AVIONIC  ELECTRONIC  EQUIPMENT 

1.  INTRODUCTION 

Testing  of  Communication,  Surveillance  and  Avionic  Electronic 
Equipment  is  conducted  to  determine  whether  the  equipment  meets  its  design 
specifications  and  is  satisfactory  for  its  intended  use.  All  equipments  are 
subjected  to  three  categories  of  tests,  Engineering,  Service,  and  Environmental 
tests.  Within  each  of  these  categories  of  tests,  equipments  are  operated  at 
various  levels  and  in  combination  with  other  equipments  (in  subsystem  or  system 
groups),  and  are  subjected  to  a  full  range  of  operational  and  environmental 
conditions . 


The  basic  objectives,  preparation,  instrumentation  techniques  and 
methodology  employed  in  developing  these  tests  are  discussed  in  the  following 
sections. 

2.  GENERAL  CONSIDERATIONS 


The  approach  to  any  development  test  inherently  involves  the  ques-. 
tions  of  what  data  shall  be  obtained  and  how  shall  they  be  obtained  to  permit 
an  effective  evaluation  of  equipment. 


Where  characteristics  and  criteria  are  provided  in  standard  documents 
such  as  QMR's  and  SDR's,  the  required  data  may  be  evident  or  may  be  derived. 
When  standards  are  not  provided  an  operational  analysis  of  the  equipment  is 
necessary  to  establish  the  significant  characteristics  and  variables  related 
to  equipment  performance  and  integrity.  Once  these  are  established,  methods  of 
obtaining  data  related  to  these  characteristics  and  variables  must  be  selected. 
An  example  of  the  establishment  of  significant  system  test  parameters  through 
operational  analysis  is  given  in  Appendix  A. 


The  general  methods  of  data  collection  include  the  following: 


a.  Instrumentation  and  simulation 

b.  Checklists 

c. .  Interviews 

d.  Direct  observations 


Development  tests  are  planned  for  a  complete  evaluation  of  the 
equipment  or  system.  Previous  valid  test  data  which  are  available  are  utilized 
to  the  extent  possible  and  duplication  of  test  effort  is  avoided  when  possible. 

Use  of  uniform  testing  procedures,  as  provided  by  the  USATECOM 
Materiel  Test  Procedures  is  encouraged  to  insure  comparable  test  data  with  past 
and  future  tests.  The  scope  and  purpose  of  the  USATECOM  Materiel  Test  Pro¬ 
cedures  are  discussed  in  section  8. 
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3.  ENGINEERING  TEST 

The  purpose  of  engineering  test  is  to  obtain  sufficient  information 
to  permit  a  technical  evaluation  and  a  determination  of  the  following: 

a.  Degree  to  which  the  technical  performance  of  the  equipment  meets 
the  requirements  of  the  Qualitative  Materiel  Requirement  (QMR) ,  Small  Develop¬ 
ment  Requirement  (SDR),  Technical  Characteristics  (TC) ,  and  design  requirements. 

b.  Relative  safety  of  the  equipment. 

c.  Technical  and  maintenance  suitability  of  the  equipment  for 
service  test. 

Data  from  an  engineering  test  is  also  used  in  further  development 
of  equipment. 

The  approach  employed  in  engineering  test  of  electronic  equipment  is 
basically  an  engineering  approach  which  includes  the  planning  for  technical 
performance  and  safety  data  collection.  The  test  is  characterized  by  con¬ 
trolled  conditions  and  elimination  of  errors  in  human  judgment  as  far  as 
possible  through  the  utilization  of  laboratory  equipment  and  techniques, 
statistical  methodology,  and  personnel  trained  in  engineering  or  Scientific 
fields. 

Laboratory  and  instrumented  field  tests  are  the  primary  methods  for 
obtaining  technical  performance  data.  Where  field  tests  are  employed,  the 
same  field  locations  should  be  used  for  similar  tests,  as  far  as  possible  to 
insure  comparable  results. 

For  electronic  equipment  which  comprises  a  system  including  untested 
components,  a  sequence  of  engineering  test  phases  is  required  as  follows: 

a.  Component  test 

b.  Intercomponent  compatibility  of  subsystem  interface  test 

c.  Subsystem  test 

d.  System  test 

The  scope  and  objective  of  each  test  phase  must  be  defined  and  their 
interdependence  clearly  established. 

Unfortunately,  the  word  system  has  many  colloquial  meanings,  some  of 
which  have  no  place  in  engineering  testing.  To  exclude  such  meanings,  the 
following  definition  is  stated: 

A  system  is  a  set  of  objects  with  relationships  between  the  objects 
and  between  their  attributes. 

Objects,  in  the  context  of  this  document  are  simply  the  parts  or 
components  of  a  system  and  are  unlimited  in  variety.  Testing  of  objects  con¬ 
sists  of  determining  the  attributes  or  properties  of  the  objects.  For  example, 
the  objects  listed  have  (among  others)  the  following  attributes: 
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a.  Switches  -  speed  of  operation,  state 

b.  Wire  -  tensile  strength,  electrical  resistance 

Relationships  tie  the  system  together.  The  many  kinds  of  relation¬ 
ships  (casual,  logical,  random,  etc.)  make  the  "system"  concept  useful.  For 
any  given  set  of  objects  it  is  impossible  to  say  that  no  interrelationships 
exist  since,  for  example,  for  a  particular  physical  system,  one  could  always 
consider  as  relationships  the  distance  between  pairs  of  objects.  The  re¬ 
lationships  to  be  considered  in  testing  a  given  set  of  objects  (i.e.  a  system 
test)  depends  primarily  on  the  system  mission,  trivial  or  unessential  relation¬ 
ships  excluded. 

If  one  or  more  components  or  associated  system  are  not  available  for 
system  test,  it  may  be  necessary  to  simulate  the  interfaces  of  these  items  to 
accomplish  the  test  as  described  in  MTP  6-1-002,  Mathematical  Model  Construction 
and  Evaluation.  Regardless  of  the  accuracy  of  the  simulation,  an  uncertainty 
will  exist  as  to  final  interface  compatibility  which  must  be  considered  in  the 
technical  evaluation  in  regard  to  reliability  of  test  results. 

4.  SERVICE  TEST 

The  purpose  of  service  test  is  to  obtain  sufficient  information  to 
permit  an  operational  evaluation  and  a  determination  of  the  following: 

a.  Degree  to  which  the  equipment  meets  the  military  characteristics 
(MC)  and  performs  the  mission  as  expressed  in  the  QMR  or  SDR. 

b.  Suitability  of  the  equipment  and  its  maintenance  package  for  use 
by  the  Army. 

These  determinations  provide  the  basis  for  recommendations  on  type 
classification  of  the  equipment. 

The  approach  employed  in  service  test  of  electronic  equipment  is 
basically  a  subjective  approach  which  includes  the  planning  for  collection  of 
military  characteristic  data  under  simulated  or  actual  field  conditions.  The 
test  is  characterized  by  qualitative  observation  and  evaluation  of  operational 
equipment  by  selected  military  personnel  having  a  background  of  field  experi¬ 
ence  with  the  type  of  equipment  undergoing  test.  Instrumentation  is  limited 
to  measurements  of  those  characteristics  which  have  major  operational  signi¬ 
ficance. 

5.  ENVIRONMENTAL  TEST 

The  purpose  of  environmental  test,  which  forms  an  integral  part  of 
engineering  and  service  test,  is  to  obtain  sufficient  information  to  determine 
whether  the  test  item  performs  effectively  in  the  environments  of  its  intended 
use. 

The  approach  employed  in  environmental  test  of  electronic  equipment 
is  consistent  with  that  of  the  overall  engineering  or  service  test.  Testing 
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in  simulated  climatic  extremes,  such  as  provided  by  environmental  test  chambers, 
is  accomplished  to  the  maximum  extent  in  the  engineering  and  prior  test  phases. 
Testing  in  extreme  natural  climatic  environments  is  used  to  substantiate  or 
supplement  data  obtained  from  simulated  tests. 

6.  INSTRUMENTATION  TECHNIQUES 

In  the  course  of  development  testing  of  communications,  surveillance 
and  electronic  equipment  a  wide-range  of  instrumentation  is  generally  required. 
These  can  be  classified  in  terms  of  the  fundamental  technology  upon  which  it 
is  based  to  include  but  not  limited  to  the  following: 

a.  Electronic 

b.  Electromechanical 

c.  Photographic 

d.  Optical 

Instrumentation  provides  the  primary  physical  means  for  obtaining 
data  on  equipment  and  environmental  characteristics.  For  example,  this 
applies  to  the  measurement  of  such  fundamental  parameters  of  electronic 
signals  as  voltage  (field  intensity),  current,  power,  waveform  (modulation), 
frequency,  period,  phase,  and  such  equipment  characteristics  which  operate 
on  the  signal  as  directivity,  gain,  impedance,  sensitivity  (noise  figure), 
selectivity  (bandwidth),  linearity,  time  constants,  attenuation  and  correlation. 

Once  characteristic  parameters  and  criteria  are  established,  the 
next  step  is  the  choice  of  a  method  of  measurement.  This  choice,  made  in 
consideration  of  available  instrumentation,  must  take  into  account  required 
accuracy  among  other  factors.  Indirect  measurement  data  must  be  reducible 
to  final  data  (variable  or  parameter)  by  data  reduction  equations  or  rela¬ 
tionships  . 

Not  only  must  the  method  be  considered  but  also  the  details  of 
procedure  must  be  carefully  planned.  The  same  method  and  instrumentation 
may  be  used  by  two  test  personnel,  yet  one  may  obtain  better  results,  the 
difference  being  in  the  care  with  which  small  disturbing  factors  are  elim¬ 
inated.  It  is  often  possible  to  plan  the  procedure  so  that  shortcomings  in 
the  instrumentation  or  stray  effects  will  balance  out  in  the  final  data. 

One  of  the  basic  principles  in  the  description  and  analysis  of 
measured  data  is  that  of  randomization.  Actual  data  are  imagined  to  be 
a  random  selection,  one  for  each  measurement,  of  values  from  a  large  reference 
distribution  which  could  be  generated  by  infinite  repetition  of  the  test.  In 
statistical  terms,  this  is  a  finite  sample  from  a  population  distribution. 

For  reasons  of  mathematical  convenience,  it  is  usual  to  assume  that  the  popu¬ 
lation  distribution  can  be  approximated  satisfactorily  by  an  analytic  function 
(distribution  function)  having  two  or  three  parameters.  A  finite  data  sample 
permits  at  most  the  assignment  of  confidence  levels  to  the  values  of  the 
distribution  function  parameters  which  represent  the  best  value  and  the  signi¬ 
ficance  of  the  measurement.  Description  of  measured  data  is  covered  in 
greater  detail  in  MTP  3-1-002,  Confidence  Intervals  and  Sample  Size. 
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Sound  instrumentation  selection  dicates  that  the  test  engineer  be 
thoroughly,  familiar  with  the  basic  principles  underlying  the  measurement 
methods  and  the  instrumentations  utilized.  In  the  final  analysis,  the  choice 
of  any  particular  method  is  restricted  by  the  availability  of  suitable  instru¬ 
mentation. 


The  capability  to  apply  instrumentation  is  not  sufficient  in  itself. 
In  order  to  provide  universal  application,  measurements  must  be  based  upon 
internationally  accepted  standards.  Moreover,  measuring  instruments  must  be 
frequently  calibrated  against  these  standards. 

7.  GENERAL  TEST  DESIGN  CONSIDERATIONS 

Once  the  significant  characteristics  and  the  means  for  measuring 
or  describing  them  (directly  or  indirectly)  have  been  determined,  a  statis¬ 
tical  analysis  and  design  can  be  developed  to  permit  the  required  analysis 
or  evaluation  in  accord  with  test  objectives.  These  may  be  categorized  as 
follows: 


a.  Comparison  between  the  standard  QMR  or  SDR  requirements  and 
equipment  performance  as  determined,  that  is,  the  accomplishment  of  the 
characteristic  or  function  and  the  degree  of  such  accomplishment. 

b.  Comparison  between  two  different  configurations  of  the  same 
equipment,  or  between  two  equipments  tested  under  similar  conditions  where 
one  may  be  the  control  item. 

c.  Correlations  among  variables. 

In  testing,  an  independent  variable,  usually  the  input  signal,  is 
the  factor  which  is  varied.  Dependent  variables  or  outputs  are  generally 
equated  with  the  equipment  or  system  response. 

In  laboratory  tests  all  the  variables  except  one  affecting  equip¬ 
ment  performance  may  be  controlled  experimentally,  that  is,  by  holding  them 
constant  or  by  varying  such  extraneous  variables  systematically.  These 
include  the  variables  due  to  environmental  factors  which  can  be  simulated  in 
various  environmental  test  chambers. 

In  field  tests,  such  as  those  required  for  radar  systems,  it  is 
usually  more  difficult  to  control  input/output  relationships.  The  multi¬ 
variate  nature  of  the  system  operation  may  require  a  type  of  statistical 

treatment  such  as  multivariate  analysis  in  which  control  can  be  exercised 

statistically.  For  field  tests,  environmental  factors  or  complete  weather 
information  must  be  recorded  on  an  "as-tested"  basis. 

Two  major  factors  which  determine  the  selection  of  a  statistical 
design  are  as  follows: 

a.  The  amount  of  pretest  control  available. 

b.  The  number  of  variables  whose  relationship  must  be  determined. 

Statistical  design  can  be  roughly  categorized  as  either  comparative 
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or  correlational,  bivariate  or  multivariate.  For  example,  comparative  bivariate 
statistics  include  Student's  "t"  and  analysis  of  variance.  Comparative  multi¬ 
variate  statistics  include  the  many  variation  of  factorial  analysis  of  variance 
and  analysis  of  co-variance.  An  example  of  a  correlational  bivariate  technique 
is  the  Pearson  "r".  Correlational  multivariate  techniques  include  multiple 
regression  analysis  or  partial  correlation.  These  statistical  techniques  are 
described  in  greater  detail  in  References  6  through  8. 

If  multivariate  techniques  cannot  provide  the  degree  of  test  control 
such  as  in  tests  of  large  electronic  systems,  then  the  approach  described  in 
MTP  6-1-002,  Mathematical  Model  Construction  and  Evaluation  in  conjunction  with 
engineering  test  can  provide  another  means  of  assessing  the  relationships  among 
many  variables. 

In  summary,  the  factors  to  be  considered  in  the  overall  design  of  a 
test  include  the  following: 

a.  Documentation  such  as  test  directive,  QMR,  SDR,  TC,  design  re¬ 
quirements,  and  previous  test  data. 

b.  Test  objective. 

c.  Equipment  characteristics  or  operations  to  be  measured  or 

described. 

d.  Test  constraints  or  limitations. 

e.  Test  variables  and  measures  (including  human  operator,  as 
applicable) . 

f.  Instrumentation. 

g  Statistical  analysis  and  design. 

h.  Data  reduction  techniques. 

i.  Test  facility  (including  extreme  natural  environments). 

j .  Equipment  operators. 

Results  of  engineering  and  service  tests  are  reported  in  test 
reports  in  accordance  with  the  test  directive  and  appropriate  Test  and 
Evaluation  Command  formats. 

8i  USATECOM  MATERIEL  TEST  PROCEDURES 

The  USATECOM  Materiel  Test  Procedures  of  which  this  document  is  a 
part  are  a  collection  of  documents  that  define  those  test  procedures  utilized 
by  the  U.  S.  Army  Test  and  Evaluation  Command  in  the  Engineering  and  Service 
testing  of  Army  materiel. 

The  USATECOM  Materiel  Test  Procedures  are  being  designed  specifi¬ 
cally  to  accomplish  the  following  objectives: 

a.  Provide  a  definitive  document  that  describes  existing  test 
procedures  for  utilization  in  the  preparation  of  Test  Directives,  Test  Plans, 
and  Test  Reports,  and  in  the  conduct  of  test  operations. 

b.  Provide  a  uniform  testing  procedure  to  insure  comparability  of 
tests  conducted  upon  the  same  commodity  item  at  different  test  activities,  or 
by  the  same  activity  at  different  times. 
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c.  Provide  Headquarters,  USATECOM  with  an  up-to-date  statement  of 
current  testing  procedures  so  that  the  procedures  may  be  reviewed  for  adfequacy 
in  term  of  the  existing  state  of  the  art. 

d.  Provide  Headquarters,  USATECOM  with  a  standard  for  the  direction 
of  the  improvement  effort  of  test  procedures. 

e.  Reduce  the  time  required  for  the  preparation,  review,  and  pro¬ 
cessing  of  Test  Plans. 

f.  Provide  developing  agencies  and  military  contractors  with  a 
detailed  knowledge  of  Engineering  and  Service  Test  requirements  in  order  to 
facilitate  coordinated  test  planning. 

g.  Serve  as. a  guide  for  developing  agencies  and  military  con¬ 
tractors  conducting  tests  on  military  commodities. 


It  is  to  be  emphasized  that  the  Materiel  Test  Procedures  are  a 
guide  to  current  testing  technology  and  are  not  regularatory  in  nature.  In 
those  cases  where  unusual  test  requirements  exist,  or  where  new  and  improved 
procedures  have  been  devised,  appropriate  testing  procedures  will  be  used  by 
the  test  activity.  However,  in  order  to  accomplish  the  stated  objectives  of 
this  program  it  is  incumbent  upon  all  users  of  these  procedures  to  make  such 
suggestions  as  required  to  insure  the  technical  accuracy,  adequacy,  and 
currency  of  the  Materiel  Test  Procedures. 


9.  TEST  FACILITIES 

Facilities  available  to  support  an  electronic  test  mission  are 
described  in  detail  in  the  USAEPG  Instrumentation  and  Test  Facilities  pamphlet 
produced  by  the  U.  S.  Army  Electronic  Proving  Ground.  A  digest  of  established 
field  sites,  laboratories  and  their  associated  work  shops,  communication, 
automatic  data  processing  and  electronic  equipments  includes: 


a. 

b. 

c. 

d. 

e. 

f. 

g. 

h. 

i. 
1. 

k. 

l. 

m. 

n. 

o. 
P- 

q- 

r. 

programs . 


An  electromagnetic  environmental  test  facility. 

An  electronic  countermeasures  vulnerability  test  facility. 

A  systems  test  facility. 

Standards  and  calibration  facility. 

Tracking  and  surveillance  sensor  testing  facility. 

Data  processing  and  reduction  facility. 

Central  communications  system. 

A  central  timing  system 
Frequency  control. 

Data  acquisition. 

Aircraft  to  support  tests. 

Antenna  test  towers  and  recording  equipment. 

Free  space  simulation  using  a  signal  source. 

Artillery  firing  ranges. 

Sensor  resolution  complexes. 

Special  engineering  and  human  factors  laboratories. 

Military  support  for  tactical  environment  testing  and  evaluation. 
Facilities  to  simultaneously  conduct  a  number  of  independent  test 
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APPENDIX  A 

OPERATIONAL  ANALYSIS  OF  AN  ELEMENTARY  COMMUNICATIONS  SYSTEM 

As  an  example  an  approach  to  the  establishment  of  significant  system 
test  parameters  through  operational  analysis,  Figure  1  shows  a  model  of  an 
elementary  communication  system.  It  contains  five  functions  which  are  necessary 
to  any  communication  process.  Certain  of  these  functions  may  become  rudimentary 
or  even  vanish  in  particular  cases.  The  sixth  element,  the  noise  source,  is 
usually  unavoidable  and  unwanted. 


Transmitted  Transmitted  Received  Received 


I  I 

l  Noise  | 

,  Source  | 

i _ ! 


Figure  1.  The  Elementary  Communications  System 


The  functions  of  the  basic  system  are  defined  as  follows: 

Source:  The  origin  of  the  message  (or  sequence  of  messages)  to  be 
sent  over  the  channel.  It  consists  of  a  population  of  possible  messages.  Its 
output  is  the  message  selected  from  the  population.  Examples:  a  deck  of 
punched  cards,  a  picture  or  scene,  a  human  voice. 

Encoder:  Operates  on  the  message  from  the  source  and  converts  it 
to  signals  that  the  channel  will  accept.  Its  output  consists  of  the  orig¬ 
inating  signals.  It  is  a  matching  device  that  resolves  incompatibilities 
between  the  source  and  the  channel.  Examples:  a  microphone  which  converts 
acoustic  to  electric  energy,  a  cryptographer  using  his  codebook  to  encipher 
a  message. 
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Signal  Channel:  The  unique  path  that  conveys  the  signals  to  the 
decoder.  Some  of  the  ways  by  which  a  particular  signal  channel  is  differ¬ 
entiated  from  all  others  are  by  space  separation  (use  of  separate  wire  or 
radio  path);  by  frequency  separation  (in  frequency-division  multiplex 
systems);  by  time  separation  (in  time-division  multiplex  systems);  by 
electrical  balance  (in  simplex  and  phantom  circuits);  by  directions  dis¬ 
crimination  (in  biconjugate  circuits);  or  by  other  means  (such  as  the  polar¬ 
ization  of  a  radio  wave) . 

Decoder:  Operates  on  the  received  signals  and  attempts  to  extract 
the  message  and  present  it  in  a  usable  form  to  the  destination.  The  output 
of  the  decoder  is  the  received  message. 

Destination:  The  recipient  and  utilizer  of  the  received  message. 

Noise  Source:  Adds  an  interfering  signal  to  the  message  signals. 
Noise  includes  thermal  noise,  acoustic  noise,  atmospheric  static,  power 
induction,  crosstalk,  etc.  There  are  actually  noise  sources  in  all  parts 
of  a  system.  The  total  effect  of  all  of  them,  however,  may  be  represented 
by  an  equivalent  signal  noise  source  in  the  signal  channel.  In  this  view, 
noise  anywhere  in  the  system  effectively  becomes  a  property  of  the  channel. 
Two  other  types  of  noise  that  are  important  but  not  yet  integrated  with 
the  theory  are  semantic  noise  and  psychological  noise.  Semantic  noise 
refers  to  the  ambiguity  inherent  in  some  messages,  such  as  those  encoded 
in  the  English  language.  Psychological  and  psychophysical  noise  may 
refer  to  several  phenomena,  one  source  arises  from  various  thresholds  of 
perception  to  physical  and  psychological  stimuli. 

For  a  communication  system  to  be  of  any  value,  at  least  part  of 
the  information  contained  in  the  initial  message  must  be  received  at  the 
destination.  But  it  may  be  altered  in  one  or  more  ways.  It  may  be  changed 
in  form,  added  to,  subtracted  from,  or  delayed  in  time  with  respect  to  the 
original  message. 

Thus  in  any  practical  communication  system  a  source  can  be 
identified  that  generates  the  message,  a  destination  that  receives  it, 
and  a  channel  that  transmits  it.  Without  scrutinizing  the  composition  of 
these  functions,  the  sources  can  be  described  in  terms  of  the  properties 
of  the  messages  it  can  generate,  the  destination  in  terms  of  the  properties 
of  the  messages  acceptable  to  it,  and  the  channel  in  terms  of  the  properties 
of  the  signals  it  can  accept  and  deliver.  Transducers  are  required  between 
source  and  channel  if  their  respective  properties  are  incompatible,  and 
similarly  between  channel  and  destination. 
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